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ABSTRACT 

Computer-based  tools  are  designed  and  implemented  to  improve  the  quality  and 
speed  of  the  analysis  of  results  from  large  scale  optimization  models.  These  tools  are 
needed  to  manage  the  voluminous  output  generated  by  these  models.  This  approach 
is  applied  to  the  United  States  Air  Force  model  HEAVY  ATTACK  which  is  a  large- 
scale  non-linear  optimization  program  used  to  plan  air-to-ground  munitions 
requirements.  A  single  execution  of  the  model  produces  about  4,500  lines  of  results. 
The  HEAVY  ATTACK  decision-making  process  is  studied  and  specific  analysis  tools  are 
designed  and  implemented  with  a  spreadsheet  program  on  a  personal  computer. 
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I.  INTRODUCTION 

Over  the  past  several  years,  the  computational  power  of  mainframes  and  personal 
computers  has  increased  manyfold.  Much  larger  and  more  complex  optimization 
models  (involving  multiple  scenarios)  can  be  solved  in  less  time.  This  implies  a 
commensurate  growth  in  the  size  of  output.  The  sheer  volume  of  output  from  these 
models  is  overwhelming;  multiple-run  comparisons  involve  even  more  output.  Current 
analysis  techniques  have  become  awkward  and  time  consuming. 

Given  sufficient  time,  detailed  output  analysis  for  almost  any  model  can  be 
accomplished.  However,  decision  makers  usually  place  time  restrictions  upon  the 
analyst.  The  scope  of  analysis  usually  must  be  narrowed  down  to  some  manageable 
size.  The  application  of  automated  analysis  tools  offers  the  potential  to  do  more 
thorough  analysis  in  less  time. 

Computer  tools  are  necessary  to  automate  the  analysis  process  and  manage  the 
vast  amount  of  data  and  output.  The  aim  of  these  tools  is  to  enhance  the  analysis 
process  by  improving  the  quality,  speed  and  scope  of  output  analysis,  and  reduce  data 
input  errors.  The  design  of  visualization  tools  to  aid  operations  research  analysis  must 
be  preceded  by  a  careful  evaluation  of  the  decision-making  process.  Applicable 
visualization  principles  [Ref.  l:p.  11-12]  must  be  incorporated  into  the  design.  There 
are  many  different  ways  to  present  information,  therefore,  design  choices  must  be 
made. 


The  focus  of  this  thesis  is  to  (1)  analyze  a  decision-making  process  and  identify 
analysis  choke  points,  (2)  design  analysis  tools,  and  (3)  demonstrate  them. 

An  important  operations  research  model,  HEAVY  ATTACK,  has  been  selected 
to  demonstrate  these  tools.  This  large-scale  optimization  model  has  been  used  by  the 
Combat  Forces  Division  at  Headquarters,  United  States  Air  Force,  for  over  two  decades 
to  generate  air-to-ground  munitions  requirements  for  the  Air  Force  Statement  of 
Requirement  in  order  to  justify  procurement  objectives.  HEAVY  ATTACK  is  widely 
accepted  by  the  Department  of  Defense  and  the  Congress,  and  is  used  to  help  plan  the 
expenditure  of  over  $2  billion  each  year.  Recently,  HEAVY  ATTACK  was  used  by  the 
Combat  Forces  Division  for  analysis  of  munitions  requirements  for  Operation  Desert 
Storm. 

Chapter  II  provides  a  general  view  of  the  HEAVY  ATTACK  model.  Chapter  III 
describes  the  current  difficulties  with  using  HEAVY  ATTACK  and  with  analyzing  its 
output.  Steps  to  improve  the  analysis  are  described.  Chapter  IV  discusses  the 
implementation  of  the  analysis  tools.  These  tools  are  based  on  spreadsheets.  Chapter 
V  states  the  conclusions. 


H.  HEAVY  ATTACK  MODEL 

For  over  25  years  mathematical  programming  has  been  the  foundation  for  the 
budget  planning  of  air-to-ground  conventional  ordnance  by  the  Combat  Forces  Division 
within  the  Directorate  of  Operations  of  the  United  States  Air  Force.  The  planning 
goal  is  to  ensure  sufficient  quantities  of  war  reserve  munitions  are  purchased  annually. 
The  Division's  Air-to-Ground  Nonnuclear  Consumables  Annual  Analysis  (A/G  NCAA) 
(Figure  2-1)  currently  uses  three  programs  -  SELECTOR,  HEAVY  ATTACK,  and 
HEAVY  GOAL  —  to  accomplish  this  goal.  These  models  reside  on  a  mainframe 
computer  in  the  Pentagon. 

A.      A/G  NCAA  METHODOLOGY 

In  the  annual  analysis  process  the  first  model,  SELECTOR,  determines  weapons 
effectiveness  based  on  the  Joint  Munitions  Effectiveness  Manual  (JMEM)  [e.g.  Ref.  2], 
weather,  aircraft  attrition  and  cost.  The  JMEM  lists  the  average  number  of  targets 
of  type  j  killed  by  sortie  type  i  using  a  specified  tactic  t  in  a  given  weather  of  type  w. 
SELECTOR  takes  the  JMEM  data  and  produces,  as  an  input  to  HEAVY  ATTACK,  the 
average  number  of  targets  of  type./  killed  by  sortie  type  i  [Ref.  3:p.  4].  The  data  is 
then  sorted  from  the  most  cost-effective  to  the  least.  Because  the  most  effective 
weapon  is  usually  the  most  expensive  or  because  there  may  be  a  high  attrition  rate  to 
the  aircraft  when  employing  a  specific  tactic,  SELECTOR  chooses  the  most  cost- 
effective  tactic. 
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Figure  2-1.  A/G  NCAA  Methodology  Overview. 
The  second  program,  HEAVY  ATTACK,  is  the  cornerstone  of  the  planning  effort. 
It  is  a  FORTRAN-based,  large-scale  optimization  program  with  a  nonlinear  objective 
function  and  linear  constraints.  For  each  of  seven  time  periods,  varying  in  duration 
from  hours  to  months,  HEAVY  ATTACK  uses  the  SELECTOR-nominated  optimum 
weapon  for  each  aircraft/target  combination,  loads  that  weapon  onto  the  aircraft,  and 
allocates  aircraft  sorties  to  each  target.  The  model's  objective  is  to  maximize  the 
expected  value  of  targets  destroyed  taking  into  account,  among  other  things,  aircraft 
attrition,    the    ability   of  targets   to    regenerate   themselves,   weather,    and   Von 

« 

Clausewitz's  "fog  of  war"  [Ref.  4].  In  essence,  HEAVY  ATTACK  fights  a  multi-period, 
theater-level  war  to  determine  the  requirements  for  ordnance  stock. 


The  Combat  Forces  Division  operates  these  models  and,  for  each  respective 
major  Air  Force  theater,  hosts  an  individual  annual  week-long  air-to-ground  NCAA 
planning  conference  at  the  Pentagon.  Senior  staff  officers  bring  their  commander's 
theater  plan  for  their  conference.  This  HEAVY  ATTACK  input  is  what  the 
commander  assesses  is  his  mission,  the  threat  he  is  facing,  the  enemy's  order  of  battle, 
and  his  war  fighting  strategy.  Other  data  input  requirements  for  HEAVY  ATTACK 
include  the  sortie  effectiveness  data,  weather,  aircraft  availability  and  weapon 
inventory.  Aircraft  availability,  provided  by  the  Combat  Forces  Division,  is  obtained 
from  the  Air  Force  War  and  Mobilization  Plan,  which  allocates  tactical  air  forces 
between  the  theaters.  Weapon  inventories  are  apportioned  among  the  theaters  by  the 
Combat  Forces  Division.  [Ref.  51  HEAVY  ATTACK  takes  these  inputs  and  produces 
the  weapon  requirements  for  each  theater. 

When  each  theater  has  completed  its  annual  visit,  their  requirements  are 
combined  to  form  an  input  to  the  last  model  -  HEAVY  GOAL.  HEAVY  GOAL  takes 
the  combined  requirement,  plus  attrition  rates  for  sea  lift  and  in-theater  ground 
transportation,  and  furnishes  the  aggregate  weapon  requirements  for  the  Air  Force 
Program  Objective  Memorandum  (POM). 

B.      HEAVY  ATTACK  OPTIMIZER 

In  1974  the  Air  Force  adopted  a  nonlinear  programming  model  written  by  Clasen, 
Graves,  and  Lu  from  RAND  Corporation  to  "select  an  air-to-surface  munitions  mix  to 
be  stockpiled  as  war  reserve  inventory."  [Ref.  6:p.  11  The  objective  was  to  maximize 
the  "military  worth"  of  targets  destroyed  by  allocating  sorties  to  targets.  Their  single 
period  munitions  mix  methodology  consisted  of  three  steps:  (1)  selecting  a  munition 


for  each  aircraft-target  combination  based  on  a  criterion  of  least-cost-per-expected- 
target-killed;  (2)  allocating  aircraft  sorties  to  targets,  with  each  sortie  carrying  the 
selected  "optimum"  munition;  and  (3)  computing  the  required  munitions  mix.  Figure 
2-2  shows  the  relationship  between  these  steps  and  various  inputs. 


NPUT 


MODEL 


OUTPUT 


USER      MPOSED 

TARGET 
AND    SOOT  I E 
CO  NST  RAISES 

SOOT  IE 

CAPAP,     L ITY       IN 
iNTEOVAL     1 


POTENT  IAL 
TABGETC  |N 
INTERVAL     t 

INDEX    OF 

BELAT  WE    TARGET 

IMPORTANCE      IN 

INTERVAL     T 


WEAPON    loa: 
FACTORS 

UNIT    PROCUREMENT 
XSTS    FOR    EAC^- 
WEAPON    T'PE 


SELECT 

OPTIMUM   WEAPON 

FOR    EACH 

TARGET/ A  I  PC RAFT 
COMBINATION 


ALLOCATE     SORTIES    TO    VARIOUS 

TARGETS    FOR    MAXIMIZING    MILITARY 

WORTH    OF     TARGETS    KILLED 


CALCULATE    MUN'TiONS 

REDU'REMENTS    fqc 

NTERvAl    t 


TARGETS   KILLED 
IN     INTERVAL     t 


C      MUNlT IONS    MIX 


Figure  2-2.   HEAVY  ATTACK  Optimization  Process.  [Ref.  5:p.  2] 

C.   INPUTS  TO  HEAVY  ATTACK 

There  are  many  sources  of  data  for  HEAVY  ATTACK  for  each  theater.  The 
commander's  theater  plan  provided  to  the  Combat  Forces  Division  is  one  major  input; 
it  is  in  the  form  of  the  Target  Data  File  described  below. 

Another  input,  the  weather  distribution,  plays  an  important  role  in  how  HEAVY 
ATTACK  allocates  aircraft  to  targets.  The  distribution  forecasts  meteorological 
conditions  for  the  theater  in  the  form  of  prior  probabilities  for  six  discrete  weather 
bands  ranging  from  foul  to  clear  weather. 


Aircraft  availability  and  weapons  inventory  constitute  the  remaining  major  inputs 
and  are  provided  to  the  theater  teams  by  the  Combat  Forces  Division. 

Nine  input  files  are  required  for  HEAVY  ATTACK  These  files  are  classified  as 
data,  exclusion,  or  other.  Currently,  all  files  reside  on  a  Pentagon  mainframe 
computer.  They  are  modified  using  the  mainframe's  text  editor.  The  data  and 
exclusion  files,  which  represent  the  majority  of  the  data,  are  described  in  detail  below. 
Since  the  "other"  files  are  easily  updated  with  the  mainframe  editor,  they  are  neither 
included  in  the  analysis,  nor  described. 
1.       Data  Files 

Targets,  aircraft  and  weapons  are  defined  in  base  data  files. 
a     Target  Data  File 

The  target  file  (Figure  2-3)  is  the  commander's  theater  plan.    There 
have  been  as  many  as  85  different  target  types. 
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Figure  2-3.  Example  of  an  unclassified  HEAVY  ATTACK  Target  Data  File. 
The  United  States  Air  Force  provided  these  numbers  from  an  unclassified  data 
set.   The  heavy  dots  indicate  a  continuation  of  data. 

(1)    Target  Name  and  Description.     The  name  may  be  up  to  four 

characters  and  the  description  up  to  15.     Two  examples  are  'ABHU  —  AC  Bkr, 

Hrd,Ugnd'  (Hardened,  Underground  Aircraft  Bunker)  and  'SA9  —  SA-9  SAM  battery' 

(a  Soviet-made  surface-to-air  missile  (SAM)  battery). 


(2)  Quantity.   The  number  of  targets  within  each  target  type. 

(3)  Target  Element.  The  number  of  elements  of  a  target  type.  For 
example,  each  of  the  40  tank  parks  (quantity  of  targets)  may  contain  five  tanks  (target 
elements). 

(4)  Time  Weighted  Target  Value.  A  value  given  to  each  target  type 
for  each  of  the  seven  time  periods.  This  value  indicates  how  important  it  is  to  the 
commander  to  kill  a  target  type  relative  to  all  other  types  for  a  given  time  period.  For 
example,  a  SAM  battery  would  be  weighted  high  in  the  fust  two  to  three  periods  and 
low  in  the  remaining.  Because  SAM's  are  a  very  high  threat  to  aircraft,  it  makes  sense 
to  destroy  them  at  the  beginning  of  the  war  in  order  to  achieve  air  superiority,  hence 
the  high  initial  value.  On  the  other  hand,  destroying  a  logistics  target  at  the 
beginning  of  the  war  does  not  make  military  sense;  therefore,  they  have  a  low  initial 
value  and  are  higher  in  the  last  few  time  periods. 

(5)  Coefficient  of  Confirmability.  This  number,  between  zero  and  one, 
symbolizes  the  ability  of  a  pilot  to  assess  his  battle  damage  to  the  target.  The 
coefficient  mathematically  expresses  Clausewitz's  "fog  of  war"  principle.  If  pilots 
cannot  confirm  kills,  or  discriminate  between  live  targets  and  destroyed  targets,  they 
will  waste  attacks  and  munitions  on  already  destroyed  targets.  The  number  one 
indicates  a  pilot  is  unable  to  confirm  the  effect  of  his  first  weapons  delivery,  while  zero 
represents  perfect  target  damage  assessment  [Ref.  7:p.  14]. 

(6)  Regeneration  Days  and  Quantity.  These  specify  the  average 
number  of  days  required  to  regenerate  a  destroyed  target  type  and  the  number  of 
replacements  the  enemy  has.  Regeneration  is  the  process  by  which  destroyed  targets 
are  replaced  with  available  spares  (a  pontoon  bridge  for  a  destroyed  spanning  bridge) 
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or  repaired  (filling  in  the  bomb  craters  on  a  runway).  Additional  information  may  be 
found  in  [Ref.  8:p.  14]. 

b.     Aircraft  Data  File 

This  file  (Figure  2-4)  contains  the  aircraft  allocated  to  the  theater  by 
the  Air  Force  War  and  Mobilization  Plan.  There  have  been  as  many  as  25  different 
types  of  aircraft. 
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Figure  2-4.   Example  of  an  unclassified  HEAVY  ATTACK  Aircraft  Data  File. 

(1)  Selector  Number.  The  SELECTOR  file  index  of  this  aircraft  type. 

(2)  Weather  Number.  The  number  one  indicates  the  aircraft  is  all- 
weather  radar  delivery  capable. 

(3)  Aircraft  Name.   Up  to  four  characters,  e.g.  F15C,  A10A. 

(4)  Category  Label.  Each  aircraft  type  is  placed  into  one  of  nine 
available  categories  for  the  output  reports.  Examples  are  FALCON  and  EAGLE.  This 
aggregation  is  used  by  HEAVY  GOAL  to  identify  collateral  support  requirements. 
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(5)  Aircraft  Category  Number.  This  identifies  the  category  of  each 
aircraft. 

(6)  Number  of  Sorties.  Identifies  the  total  number  of  sorties  each 
aircraft  type  is  capable  of  flying  for  each  time  period. 

(7)  Attack  Mission  Percentage.  The  percentage  of  sorties  devoted 
solely  to  air-to-ground  attack  missions  for  each  time  period. 

c.      Weapon  Data  File 

All  theaters  use  the  same  weapon  data  base;  there  have  been  as  many 
as  65  weapon  types  (See  Figure  2-5).  For  planning  purposes  the  allocation  of  weapons 
among  the  theaters  is  decided  by  the  Combat  Forces  Division. 


WFN# 

IDENT 

DESCRIPTION 

P/NP 

WPN  COST 

MAX  EXPEND 

1 

WP01 

WEAPON  1 

0 

0 

7710. 

2 

WP02 

WEAPON  2 

0 

0 

547. 

3 

WP03 

WEAPON  3 

0 

0 

41284. 

4 

WP04 

WEAPON  4 

0 

0 

0. 

5 

WP05 

WEAPON  5 

0 

0 

5002. 

• 

• 

• 

• 

• 

• 

65 

WP65 

WEAPON  65 

0 

0 

270. 

Figure  2-5.   Example  of  an  unclassified  HEAVY  ATTACK  Weapon  Data  File. 

(1)  Weapon  Number.  A  numeric  index. 

(2)  Weapon  Name  and  Description.   The  name  may  be  up  to  four 
characters  and  the  description  up  to  25.  An  example  is  LGB  —  Laser  Guided  Bomb. 

(3)  Play/No  Play.  This  is  a  binary  switch  (zero,  one).  The  number  one 
removes  the  weapon  from  HEAVY  ATTACK 

(4)  Maximum  Expenditure.    States  the  inventory  allocated  to  the 
theater. 
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2.       Exclusion  Files 

Not  every  aircraft/target/weapon  combination  is  desirable  or  feasible  in  a 
sortie  plan.  Some  convenient  method  is  necessary  to  exclude  particular  combinations. 
Of  the  possible  combinations,  HEAVY  ATTACK  accepts  inadmissable  lists  of  aircraft 
versus  targets,  aircraft  versus  weapons,  and  aircraft/targets  versus  weapons.  These 
exclusion  files  are  used  to  preclude  combinations  which  are  not  militarily  feasible. 

The  data  in  each  exclusion  file  is  in  pairs  of  rows.  For  example,  in  the 
Aircraft  versus  Target  Exclusion  File  shown  in  Figure  2-6,  the  first  row  in  each  pah- 
is  the  aircraft  type.  The  following  row  contains  the  target  types  which  will  never  be 
attacked  by  its  paired  aircraft  type. 


1 
1 

1 

26 
1 

75 
2 

1 
• 

2 

3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 

27 

28  29  38  40  41  45  47  51  56  57  59  60  61  62  63  64  66  67  69  71  72  73  74 

76 

77  78  79  81  85 

2 

3   4   5   6   7   8  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25  26 

• 
13 

1 

2 

3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20  21  22  23  24  25 

13 

26 

27 

28  29  38  40  45  47  56  59  60  61  62  63  64  66  67  68  69  70  71  72  73  74  75 

13 

76 

77 

78  79  80  81  82  83  84  85 

Figure  2-6.   Example  of  a  HEAVY  ATTACK  Aircraft  versus  Target  Exclusion 
File.   Length  for  the  unclassified  data  set  is  48  lines. 


The  Aircraft  versus  Weapon  Exclusion  File  is  similar,  except  the  numbers 
on  the  following  row  of  each  pair  are  the  weapon  types  which  will  never  be 
expended  by  its  corresponding  aircraft  type. 
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The  remaining  exclusion  file  is  the  Aircraft/Target  versus  Weapon 
Exclusion  File  shown  in  Figure  2-7.    Here,  the  first  row  of  each  pair  contains  the 
aircraft  type  followed  by  the  target  type.  The  following  row  is  the  weapon  types  which 
will  never  be  expended  by  that  aircraft  against  that  target. 


1,  5 

2   A 

6 

8   9  10  11  12  1A  15  17  20  23  2A  25  3A  A2  A6  53  55 

1,  5 

57  58 

1,12 

2   A 

6 

8   9  10  11  12  1A  15  17  20  23  2A  25  26  27  3A  38  A6 

1,12 

A7  51 

53 

55  57  58 

• 

• 

99,85 

1   2 

3 

A   5   6   7   8   9  10  11  12  1A  15  18  19  23  2A  A2  A6 

99,85 

47  55 

57 

58 

Figure  2-7.    Example  of  a  HEAVY  ATTACK  Aircraft/Target  versus  Weapon 
Exclusion  File.   Length  for  the  unclassified  data  set  is  244  lines. 


D.      OUTPUT  FROM  HEAVY  ATTACK 

HEAVY  ATTACK  produces  three  reports:  short,  medium,  and  long.  The  Combat 
Forces  Division  analyst  mainly  uses  the  short  report  (Figure  2-8).  It  is  divided  into  two 
sections:  results  for  each  time  period  and  then  cumulative  results  through  all  seven 
periods.  In  the  first  section,  data  concerning  weapons  expended  by  aircraft  category 
and  target  types  destroyed  are  listed  for  each  time  period.  The  second  contains  the 
same  data  as  the  first,  accumulated  through  the  time  periods,  plus  the  distribution  of 
weapon  types  to  target  types.  The  final  data  table  listed  in  the  report  is  the 
cumulative  and  maximum  allowable  expenditure  of  weapons  used  during  the  "war". 

The  medium  report  is  also  divided  into  two  sections:  results  for  each  time  period 
and  cumulative  results.  The  first  section  lists  the  expected  kills  of  targets  by  aircraft, 
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WEAPON 

TOTAL 

END  OF  PERIOD   1 
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-CAT  D-- 

-CAT  E- 
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• 

• 
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0 

0 

0 
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0 

53 

TARGET 

RECONSTITUTION  AND 

RESUPPLY 

END  OF 

PERIOD   1 

TARGET 

STARTING 

PRIOR 

NEW 

FRAC 

TARGETS 

CUM 

NEWLY 

CUM 

RESULT 

TYPE 

TARGETS 

TARGETS 

TARGETS 

FIXED 

FIXED 

FIXED 

KILLED 

KILLED 

TARGETS 

1  TGOl 
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16. 

0. 

0.A51 

0. 

0. 
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16 

0. 

2  TG02 

8. 

8. 

0. 

0.777 

0. 

0. 

0. 

0 

8. 

3  TG03 

32A. 

32A. 

0. 
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0. 

0. 

0. 

0 

32A. 

A  TGOA 

162. 

162. 

0. 

0.777 

0. 

0. 

0. 

0 

162. 

• 

• 

• 

• 

• 

• 

85  TG85 

0. 

0. 

0. 

1.000 

0. 

0. 

0. 

0 

0. 

TARGET 

TOTAL   TARGETS 

PERCENT 

SORTIES 

KILLED 

KILLED 

POT  KILL 

POT  VALUE 

TGOl 

A0 

16.00 
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16.0 

22A.0 

TG08 
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10.36 

A9 

10.9 

272.6 

TG16 
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20.65 

9A 

22.0 
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TG18 

• 
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A2.00 
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A2.0 

8A0.0 

• 

• 
TG79 

21 

2A.00 
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2A.0 

• 
360.0 

TARGET/WEAPON  SUMMARY  FOR  TIME  PERIOD (S)  1-  1 

TARGET  WP01  WP02 

WP03  WPOA  WP05 

WP06  WP07 

WP08 

WP09  WP10 

WP11 

WP12 

TG08 

558     0 

0 

0     0 

0 

0 

0 

0 

0 

0 

0 

TG18 

0     0 

0 
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0   882 

0 

0 

0 

0 

0 

TG19 
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0 

0     0 

0 

0 

0 

0 

A19 
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0 

TG22 

0     0 

0   A527     0 

0 

0 

0 

0 

0 

0 

0 

• 

• 

• 

• 

• 

• 

TG79 

56     0 

0 

0     0 

0 

0 

0 

0 

0 

0 

0 

WEAPON 

CUMULATIVE 

MAXIMUM 

TYPE 

EXPENDITURE 

EXPENDITURE 

WP01 

7710 

7710 

WP02 

0 

5A7 

WP03 

A128A 

A128A 

WPOA 

23551 

9999999 

• 

• 

• 

• 

WP38 

270 

270 

Figure  2-8.    Example  of  a  HEAVY  ATTACK  Short  Report.    Length  for  the 
unclassified  data  set  is  1500  lines. 


the  allocation  of  aircraft  sorties  to  targets,  and  the  expenditure  of  weapons  by  aircraft. 
The  second  part  accumulates  the  same  aircraft  allocation  and  weapons  expenditure 
data  (see  Figure  2-9). 
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Figure  2-9.   Example  of  a  HEAVY  ATTACK  Medium  Report.   Length  for  the 
unclassified  data  set  is  2700  lines. 

The  long  report,  typically  over  25,000  lines,  contains  all  the  information  of  the 
short  and  medium  reports,  plus  additional  information  produced  by  the  optimizer. 
Some  additional  information  is  evaluated  by  the  analyst  and  the  theater  team,  but  the 
majority  of  the  information  is  useful  only  in  diagnosing  any  data  errors  identified  by 
the  optimization. 
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m.  HEAVY  ATTACK  DECISION-MAKING  PROCESS 

Analysis  of  HEAVY  ATTACK  results  is  a  tedious  process.  HEAVY  ATTACK 
generates  a  tremendous  amount  of  output.  Because  there  is  only  one  week  for  the 
planning  conference  the  theater  team  scarcely  has  time  for  a  thorough  and  in-depth 
analysis.  In  addition,  multiple  runs  of  HEAVY  ATTACK  are  required  in  order  to 
generate  acceptable  results,  further  exacerbating  the  problem. 

A.  STEPS  TOWARD  A  SOLUTION 

Three  steps  are  employed  —  analyze  the  decision-making  process,  design  analysis 
tools,  and  demonstrate  them.  The  first  step  examines  how  the  theater  teams  and 
current  Combat  Forces  Division  analyst  and  operator  of  HEAVY  ATTACK,  MAJ  Lance 
Buckingham,  USAF,  make  their  decisions.  Analysis  choke  points  are  identified. 
Second,  a  design  is  proposed  to  improve  the  time  and  quality  of  analysis.  Finally,  this 
design  proposal  is  demonstrated  with  prototypic  computer-based  tools.  A  scenario  will 
serve  to  illustrate  the  design. 

B.  THE  DECISION-MAKING  PROCESS 

Figure  3-1  shows  the  HEAVY  ATTACK  decision-making  process.  When  a  theater 
team  arrives  at  the  Pentagon  for  their  planning  conference,  the  first  step  in  the 
decision-making  process  is  to  create  the  HEAVY  ATTACK  input  files.  These  files  are 
manually  reviewed  for  correctness.  The  Best  Munitions  Table  Generator  program  is 
used  to  assist  in  reviewing  the  exclusion  files.     Errors  in  the  exclusion  files  are 
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INPUT 


Figure  3-2.   NCAA  Decision-Making  Process 

corrected,  and  the  Generator  program  is  re-run.   This  cycle  continues  until  the  files 
appear  error-free. 

All  input  files  are  then  passed  to  HEAVY  ATTACK,  and  the  model  is  run.  The 
Combat  Forces  Division  analyst  evaluates  the  output  to  validate  the  model.  The 
analyst  will  then  present  the  model's  results  to  the  team,  who  will  decide  if  any 
adjustments  need  to  be  made  to  the  input  files.  The  files  are  updated  and  the  model 
is  re-run.  This  cycle  continues  until  the  team  achieves  their  final  results.  The  team 
may  also,  during  this  cycle,  send  the  updated  exclusion  files  back  into  the  Table 
Generator  program  to  re-validate  the  tables.  When  the  team  is  satisfied  with  the 
results  from  HEAVY  ATTACK,  their  weapon  requirements  are  combined  with 
requirements  from  the  other  theaters  an  an  input  into  HEAVY  GOAL. 
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Two  major  analysis  problems,  or  choke  points,  impede  decision-making.  First  is 
the  Table  Generator  program,  which  takes  significant  time  to  run  and  to  evaluate. 
The  second  is  the  actual  analysis  of  HEAVY  ATTACK  Each  of  these  choke  points  is 
amenable  to  automated  analysis  tools. 
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IV.  ANALYSIS  TOOLS  FOR  HEAVY  ATTACK 

In  order  to  see  how  to  improve  the  analysis  of  HEAVY  ATTACK,  one  must 
understand  how  the  analyst/theater  teams  presently  use  the  model.  Figure  4-1  is  a 
current  HEAVY  ATTACK  Aircraft  Data  File.    There  is  an  obvious  problem:  this  file 
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Figure  4-1.   HEAVY  ATTACK  Aircraft  Data  File. 

contains  a  lot  of  numbers  with  very  little  associated  context.  Because  HEAVY 
ATTACK  is  a  FORTRAN  program,  its  output  may  be  influenced  by  a  single  mis-aligned 
input  field.  The  Target  and  Weapon  Data  Files  (Figures  2-4  and  2-5)  also  exhibit  this 
weakness. 

The  exclusion  files  also  present  major  problems.  Figure  4-2  is  a  restatement  of 
the  Aircraft/Target  versus  Weapon  Exclusion  File.  As  represented  by  the  figure,  these 
files  are  all  numbers  with  no  associated  context.  For  example,  the  first  and  second 
lines  of  Figure  4-2  exclude  aircraft  type  1  from  attacking  target  type  5  with  weapon 
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Figure  4-2.     Example  of  an  unclassified  HEAVY  ATTACK  Aircraft/Target 
versus  Weapon  Exclusion  File. 

types  2,  4,  6,  and  so  on.  The  exclusion  files  are  the  major  source  of  errors  with  the 
input.  Because  of  the  difficulty  in  identifying  errors  in  these  files,  the  Table 
Generator  program  was  created. 

After  HEAVY  ATTACK  is  run,  the  output  reports  are  generated.  Figure  2-8  is 
just  a  portion  of  an  unclassified  Short  Report,  which  contains  information  for  each  of 
the  seven  time  periods  plus  accumulation  across  the  seven  periods.  The  week-long 
planning  conference  provides  little  time  to  thoroughly  analyze  the  HEAVY  ATTACK 
reports. 

A.      ANALYSIS  IMPROVEMENTS 

The  next  step  was  to  design  a  computer  system  which  might  provide  analysis 
improvements.  Several  commercially  available  programs  were  reviewed  and  Borland 
International's  Quattro  Pro  3.0  spreadsheet  program  was  chosen.  Quattro  Pro  provides 
superior  graphic  capabilities  [Ref.  9,10,11,121  and  "prints"  to  text  (ASCII)  files.  This 
text  file  capability  is  extremely  important  in  order  to  transfer  data  back  and  forth 
between  desktop  computers  (Quattro  Pro)  and  mainframes  (HEAVY  ATTACK). 
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Further,   Quattro  Pro  provides  the  ability  to  import  a  text  file  and  parse  the 
information  into  usable  spreadsheet  cells. 

The  Quattro  Pro-based  analysis  tool  has  been  named  QuikDraw.    It  is  divided 
into  two  sections:  Data  Entry  and  Output  Listing. 
1.       Data  Entry 

The  Data  Entry  handles  the  data  inputs  for  HEAVY  ATTACK  - 
specifically,  the  data  and  exclusion  files  described  in  Chapter  II.  The  reniaining  "other" 
files  are  updated  by  the  Combat  Forces  Division  analyst  using  the  mainframe 
computer's  text  editor.  The  HEAVY  ATTACK  Target  and  Weapon  Data  Files  are 
combined  into  a  single  Quattro  Pro  worksheet.  The  Aircraft  Data  File  has  its  own 
Quattro  Pro  worksheet  (Figure  4-3).  (The  term  worksheet  is  synonymous  herein  with 
file). 

Currently  HEAVY  ATTACK  lists  combinations  of  aircraft,  targets,  and 
weapons  which  are  militarily  infeasible  or  invalid.  However,  the  theater  teams  tend 
to  think  in  terms  of  valid  combinations.  Therefore,  QuikDraw  has,  for  each  HEAVY 
ATTACK  exclusion  file,  a  corresponding  inclusion  worksheet.  Now,  the  users  indicate 
with  X's  the  valid  combinations  in  the  worksheets.  Figure  4-4  shows  the  unclassified 
Aircraft  versus  Target  Inclusion  Worksheet. 

The  Best  Munitions  Table  Generator  program,  which  is  the  first  choke 
point,  is  replaced  by  the  inclusion  worksheets.  Hence,  there  is  no  need  to  execute  the 
Generator  program  and  analyze  the  generated  tables. 


20 


AC 
# 

1 
2 
3 


25 


Use  SAB  WEAX  A/C 
Name   ?   #   #   CAT 


PERIOD  1 
MSNS    PCT 


PERIOD  2 
MSNS    PCT 


PERIOD  3 
MSNS   PCT 


PERIOD  4 
MSNS    PCT 


AC1A 

AC2 

AC5A 

• 


888  0.92  1221 
331  0.92  391 
205   0.92   255 


0.92 
0.92 
0.92 


1008 
309 
368 


0.92 
0.92 
0.92 


1747 
510 
630 


0.92 
0.92 
0.92 


PERIOD  7 
MSNS   PCT 

0   0.00 

»•   0   0.00 

0   0.00 

• 


AIRCRAFT 
CATEGORIES 

1  --AC1A-- 

2  --AC2-  — 

9  --AC4--  - 

Aircraft  # 

Name 

Use  ? 

SAB  # 

WEAX  # 

Cat 

Missions 

Percent 

Category 

Weather  Band 


EXPECTED  WEATHER  BAND  DISTRIBUTION 


0.05   0.20 
worst 


0.22   0.14   0.08 


0.31 
best 


Just  a  numbering  of  aircraft  from  1  to  25 

Limited  to  4  characters 

Tells  HEAVY  ATTACK  whether  to  consider  this  aircraft  type 

Index  to  the  SELECTOR  file 

1— Indicates  all-weather  radar  delivery  capable.  2-Something  less 

Indicates  which  category  the  aircraft  belongs  to  for  output  purposes 

The  total  number  of  missions.  This  includes  attack  and  air-to-air 

The  percent  of  total  missions  solely  devoted  to  air-to-ground  attack 

A  label  which  aggregates  aircraft  for  the  different  HEAVY  ATTACK  reports 

The  probability  distribution  of  weather  across  6  bands 


Figure  4-3.   Example  of  an  unclassified  QuikDraw  Aircraft  Data  Worksheet. 
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Figure  4-4.     Example  of  an  unclassified  QuikDraw  Aircraft  versus  Target 
Worksheet. 
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After  a  user  makes  changes  to  any  Data  Entry  worksheet  and  is  ready  to 
re-run  HEAVY  ATTACK,  he  executes  a  Quattro  Pro  macro  program  embedded  in  that 
worksheet.  That  macro  will  write  the  worksheet  data  to  a  text  file.  (For  simplicity, 
the  macro  is  executed  with  the  same  keystroke  in  each  worksheet).  Then,  a 
FORTRAN  program  is  executed  to  convert  the  text  files  into  the  format  required  for 
HEAVY  ATTACK  This  is  necessary  because  HEAVY  ATTACK,  as  a  FORTRAN 
program,  requires  its  input  data  to  be  in  specific  columns. 
2.       Output  Listing 

The  Output  Listing  section  has  four  worksheets;  they  are  listed  in  Table 
I.  Within  each  worksheet  are  three  graph  types:  single  run,  multiple  run,  and 
cumulative.  The  single  run  graphs  display  information  for  the  current  run;  multiple 
graphs  present  information  from  two  successive  runs.  The  cumulative  graphs  display 
"course  of  the  war"  information.  All  graphs  display  percentages,  with  raw  numbers 
available  below  the  graphs. 


Table  I.  QUIKDRAW  OUTPUT  WORKSHEETS.  THESE  WORKSHEETS  ARE 
CAPABLE  OF  GENERATING  THREE  TYPES  OF  GRAPHS.  EXAMPLES  OF  THESE 
GRAPH  TYPES  ARE  SHOWN  IN  FIGURES  4-5  THROUGH  4-8. 


Aircraft  Attack  Missions  Flown  by  Time  Period 
Weapon  Types  Expended  by  Time  Period 
Weapon  Types  Expended  across  Time  Periods 
Target  Types  Destroyed  across  Time  Periods 
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There  is  a  fundamental  difference  between  the  "by  time  period"  and  the 
"across  time  period"  worksheets.  For  the  former  the  graphed  values  are  the 
percentages  for  each  time  period.  Thus,  the  graphed  values  sum  to  100  percent.  In 
the  latter  case,  the  graphed  values  are  the  percentages  over  all  time  periods,  and  may 
exceed  100  percent  due  to  target  regeneration. 

A  special  HEAVY  ATTACK  report  (DWUNCL)  was  created  to  export  data 
to  QuikDraw.  This  report  contains  the  data  necessary  to  generate  the  graphs,  plus 
some  specific  syntactic  semaphores  for  the  Quattro  Pro  parsing  function.  The  parsing 
semaphores  allow  Quattro  Pro  to  identify  only  the  necessary  columns  from  a  report 
table,  significantly  reducing  the  amount  of  data  kept  in  the  worksheets. 

Figures  4-5  and  4-6,  Attack  Missions  Flown  and  Weapons  Expended,  are 
examples  of  "by  time  period"  graphs.  With  these  graphs  the  analyst  and  the  theater 
team  can  instantly  see  how  the  aircraft  were  allocated,  or  weapons  expended  in  time 
period  one.  This  analysis  takes  minutes  rather  than  the  tens  of  minutes  or  hours 
required  to  analyze  the  current  reports.   In  addition  this  analysis  is  more  thorough. 


23 


Period  1 


VwWwW  ... .  ..        ,  ■*- v  .*.  -  .***.*■.--*■ 


10 


15 
Percent 


20 


25 


30 


Figure  4-5.  QuikDraw  current  run  graph  of  Aircraft  Attack  Missions  Flown  by 
Time  Period  for  time  period  1  using  the  unclassified  data  set.  Percentages  sum 
to  100. 
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Figure  4-6.  QuikDraw  current  run  graph  of  Weapons  Expended  By  Time 
Period  for  time  period  1  using  the  unclassified  data  set.  Percentages  sum  to 
100. 
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Another  feature  of  QuikDraw  is  the  ability  to  view  multiple  runs.  Figure 
4-7  is  an  example  of  a  multiple  run  graph,  showing  how  the  expenditure  of  weapons 
is  affected  by  removing  an  aircraft  from  the  data  set.  To  produce  this  graph,  HEAVY 
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Figure  4-7.  QuikDraw  multiple  run  graph  of  Weapons  Expended  Across  Time 
Periods  for  time  period  1  using  the  unclassified  data  set. 


ATTACK  fust  executed  with  thirteen  aircraft  types  in  the  data  set.  The  weapons 
expended  by  those  aircraft  are  represented  by  light-shaded,  double  cross-hatch  bars 
labeled  Last  (run).  Aircraft  AC6,  having  flown  about  thirteen  percent  of  the  missions 
in  the  first  run,  was  removed  from  the  data  set  and  the  model  re-run.  The  weapon 
types  expended  by  the  remaining  twelve  aircraft  are  represented  by  the  heavy-hashed 
bars  labeled  Current  (run).  The  analyst/theater  teams  are  then  able  to  analyze  how 
removing  an  aircraft  from  HEAVY  ATTACK  affect  the  expenditure  of  weapons.  With 
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QuikDraw  the  analysis  takes  minutes,  rather  than  the  hours  analyzing  the  HEAVY 
ATTACK  reports  by  hand. 

Finally,  QuikDraw  allows  the  analyst/theater  team  to  view  cumulative 
results  over  "the  course  of  the  war"  (all  seven  time  periods).  An  example  is 
Figure  4-8  showing  how  the  target  data  base  was  destroyed  during  the  "war".  Here  five 
of  38  targets  had  higher  than  100  percent  destuction  because  of  HEAVY  ATTACK'S 
target  regeneration  feature. 

B.      TESTING  AND  IMPLEMENTATION 

QuikDraw  was  tested  using  three  different  subsets  from  an  unclassified  data  set 
provided  by  the  United  States  Air  Force  Combat  Forces  Division.  The  tests  generated 
high  confidence  that  Data  Entry  worksheets  provide  HEAVY  ATTACK  with  relatively 
quick-entry  and  error-free  input.  The  tests  also  generated  all  three  graph  types  for 
the  four  Output  Listing  worksheets. 

QuikDraw  consists  of  12  files,  24  graphs,  nearly  178,000  spreadsheet  cells,  and 
1,100  lines  of  Quattro  Pro  macro  coding  using  nearly  1,900  macro  instructions.  The 
FORTRAN  code  to  convert  the  QuikDraw  Data  Entry  worksheets  into  the  HEAVY 
ATTACK  input  files  consists  of  581  lines.  QuikDraw  is  available  from  the  author  or 
Professor  Bradley. 
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Figure  4-8.  QuikDraw  cumulative  graph  of  Targets  Destroyed  Across  Time 
Periods  for  time  periods  1—7  using  the  unclassified  data  set.  Total  percentages 
may  exceed  100%  of  available  target  types  due  to  regeneration. 
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V.  CONCLUSION 

HEAVY  ATTACK  is  an  important  operations  research  model.  It  has  been  used 
for  over  20  years  and  may  continue  to  be  used  in  the  foreseeable  future.  The 
improvements  in  analyzing  this  model  offer  significant  savings  in  man-hours  and 
money. 

HEAVY  ATTACK  is  just  one  example  of  using  a  spreadsheet  as  a  tool  to  analyze 
output.  The  approach  developed  here  to  (1)  analyze  a  decision-making  process,  (2) 
design  analysis  tools,  and  (3)  demonstrate  them,  can  be  applied  to  other  large-scale 
optimization  models. 

This  thesis  designed  and  implemented  a  state-of-the-art  output  analysis  system 
that  demonstrates  analysis  choke  points  can  be  reduced  or  eliminated.  The  current 
cumbersome,  time-consuming  data  input  and  quality  assurance  process  is  replaced  by 
QuikDraw  which  does  the  same  work  in  significantly  less  time,  reduces  errors  in  the 
input  files,  and  minimizes  the  time  spent  analyzing  hundreds  of  sheets  of  output 
through  the  use  of  graphics.   In  total  this  improves  the  quality  of  analysis. 

A  preliminary  version  of  QuikDraw  was  demonstrated  to  the  United  States  Air 
Force  Combat  Forces  Division  in  Washington  D.C.  in  December  1991.  As  a  result  of 
the  demonstration,  they  intend  to  incorporate  QuikDraw  into  the  next  generation 
NCAA  process,  which  will  be  based  on  a  486  personal  computer. 
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